UNITED STATES

SECURITIES AND EXCHANGE COMMISSION

Washington, D.C. 20549

SCHEDULE 13D

Under the Securities Exchange Act of 1934

(Amendment No. 6)*

(Name of Issuer)

(Title of Class of Securities)

(CUSIP Number)

(Name, Address and Telephone Number of Person Authorized to Receive Notices and Communications)

(Date of Event which Requires Filing of this Statement)

If the filing person has previously filed a statement on Schedule 13G to report the acquisition that is the subject of this Schedule 13D, and is filing this schedule because of §§240.13d-1(e), 240.13d-1(f) or 240.13d-1(g), check the following box.  ¨

Note: Schedules filed in paper format shall include a signed original and five copies of the schedule, including all exhibits. See §240.13d-7 for other parties to whom copies are to be sent.

The information required on the remainder of this cover page shall not be deemed to be “filed” for the purpose of Section 18 of the Securities Exchange Act of 1934 (“Act”) or otherwise subject to the liabilities of that section of the Act but shall be subject to all other provisions of the Act (however, see the Notes).

Preliminary Note

This Amendment No. 6, dated February 18, 2016 (this “Amendment No. 6”), supplements and amends the Schedule 13D filed on October 12, 2010 (as amended and supplemented to date, the “Schedule 13D”) relating to the common stock, par value $0.01 per share (the “Common Stock”), of Hampton Roads Bankshares, Inc. (the “Issuer”). Capitalized terms used in this Amendment No. 6 and not otherwise defined herein shall have the same meanings ascribed to them in the Schedule 13D.

Item 4. Purpose of the Transaction

Item 4 is hereby amended and supplemented by the following:

On February 10, 2016, the Issuer entered into an Agreement and Plan of Reorganization (the “Reorganization Agreement”) with Xenith Bankshares, Inc., a Virginia corporation (“Xenith”). The Reorganization Agreement provides for, among other things, the merger of Xenith and the Issuer, subject to the terms and conditions set forth therein. Under the Reorganization Agreement, each issued and outstanding share of common stock of Xenith shall be converted into 4.4 shares of Common Stock. Following the consummation of the transactions contemplated by the Reorganization Agreement, Carlyle Financial Services Harbor, L.P. (“Carlyle”) will own approximately 18.4% of the outstanding Common Stock of the Issuer.

On February 10, 2016, Carlyle, in its capacity as a shareholder, entered into a voting agreement with Xenith (the “Voting Agreement”), with respect to, among other things, the voting of shares of Carlyle’s Common Stock in favor of the approval of the terms of the Reorganization Agreement and the transactions contemplated thereby.

Pursuant to the Voting Agreement, Carlyle has agreed, among other things, (i) to vote or exercise its right to consent with respect to all of its shares of Common Stock in favor of the approval of the terms of the Reorganization Agreement and all agreements and actions related thereto, and (ii) not to vote in favor of, or consent to, and will vote against and not consent to, the approval of any (a) HRB Acquisition Proposal (as that term is used therein), (b) corporate action the consummation of which would reasonably be expected to frustrate the purposes, or prevent or materially delay the consummation of, the transactions contemplated by the Reorganization Agreement or (c) any other matter relating to, or in connection with, any of the foregoing matters. In connection with the foregoing, Carlyle granted a revocable proxy appointing Xenith as such attorney-in-fact and proxy, with full power of substitution, for and in such shareholder’s name, to vote, express consent or dissent, or otherwise to utilize such voting power in the manner contemplated by the Voting Agreement as Xenith or its proxy or substitute shall, in Xenith’s sole discretion, deem proper.

In addition, Carlyle shall not, without the prior written consent of Xenith, directly or indirectly, (i) grant any proxies or enter into any voting trust or other arrangement with respect to the voting of any of their shares of Common Stock relating to the matters set forth in the Voting Agreement (other than a proxy granted to any directors or officers of the Issuer), (ii) revoke the proxy granted pursuant to the Voting Agreement except upon termination of the Voting Agreement or (iii) transfer, encumber or otherwise dispose of, or enter into any contract, option or other arrangement or understanding with respect to the direct or indirect transfer, encumbrance or other disposition of, any shares of Common Stock, prior to the termination of the Voting Agreement.

Carlyle agreed, in its capacity as a shareholder of the Issuer, that it will not, and will instruct and use reasonable best efforts to cause its representatives and partners not to, directly or indirectly, (i) initiate, solicit, knowingly encourage or knowingly facilitate inquiries or proposals with respect to a HRB Acquisition Proposal, (ii) engage or participate in any negotiations with any person concerning any HRB Acquisition Proposal, (iii) provide any confidential or nonpublic information or data to, have or participate in any discussions with or otherwise cooperate in any way with, any person in connection with any HRB Acquisition Proposal or (iv) unless the Voting Agreement has been terminated in accordance with its terms, enter into any term sheet, letter of intent, memorandum of understanding, agreement in principle, acquisition agreement, merger agreement or other agreement (other than a confidentiality agreement referred to and entered into in accordance with the Reorganization Agreement) relating to any HRB Acquisition Proposal, in each case, except to the extent that the Issuer is permitted to take such action pursuant to the Reorganization Agreement. Carlyle will promptly (within 24 hours) advise Xenith following receipt of any HRB Acquisition Proposal or any inquiry which could reasonably be expected to lead to a HRB Acquisition Proposal, and the substance thereof (including the material terms and conditions of the HRB Acquisition Proposal), and will keep Xenith apprised of any related material developments, discussions and negotiations on a reasonably current basis, including any amendments to or revisions of the terms of such inquiry or HRB Acquisition Proposal, in each case to the extent HRB has not previously notified Xenith.

The Voting Agreement shall terminate upon the earlier of (a) the effective time of the reorganization, (b) termination of the Reorganization Agreement in accordance with its terms or (c) if the Issuer’s Board of Directors has delivered to its shareholders a recommendation change in accordance with the Reorganization Agreement. Upon any such termination all rights or obligations of the parties under the Voting Agreement shall immediately terminate.

The foregoing description of the Voting Agreement does not purport to be complete and is qualified in its entirety by reference to the full text of such agreement filed as exhibits to this Schedule 13D, and incorporated herein by reference.

Other than as described in this Item 4, none of the Reporting Persons currently has any plans or proposals that relate to, or would result in, any of the matters listed in Items 4(a)–(j) of Schedule 13D, although, depending on the factors discussed herein, the Reporting Persons may change their purpose or formulate different plans or proposals with respect thereto at any time.

1

Item 6. Contracts, Arrangements, Understandings or Relationships With Respect to Securities of the Issuer

Item 6 is hereby amended and supplemented by the following:

Item 4 above summarizes certain provisions of Voting Agreement and is incorporated herein by reference. A copy of the Voting Agreement is attached as an exhibit to this Schedule 13D, and is incorporated by reference herein.

Except as set forth herein, none of the Reporting Persons or Related Persons has any contracts, arrangements, understandings or relationships (legal or otherwise) with any person with respect to any securities of the Issuer, including but not limited to any contracts, arrangements, understandings or relationships concerning the transfer or voting of such securities, finder’s fees, joint ventures, loan or option arrangements, puts or calls, guarantees of profits, division of profits or losses, or the giving or withholding of proxies.

Item 7. Material to be Filed as Exhibits

Item 7 of the Schedule 13D is hereby amended and supplemented by adding the following:

2

SIGNATURE

After reasonable inquiry and to the best of my knowledge and belief, I certify that the information set forth in this statement is true, complete and correct.

Dated: February 18, 2016

Exhibit 10

VOTING AGREEMENT

This VOTING AGREEMENT (this “Agreement”), dated as of February 10, 2016 by and among Xenith Bankshares, Inc., a Virginia corporation (“Xenith”), and the undersigned (the “Shareholder”).

WHEREAS, in order to induce Xenith to enter into an Agreement and Plan of Reorganization, dated as of the date hereof (the “Merger Agreement”), by and between Hampton Roads Bankshares, Inc., a Virginia corporation (“HRB”) and Xenith, Xenith has requested Shareholder, and Shareholder has agreed, to enter into this Agreement with respect to all Shares of common stock, par value $0.01 per share, of HRB that such Shareholder beneficially owns and has voting power with respect to (the “Shares”) (as used herein, the term “Shares” shall mean (i) all shares of HRB common stock or other capital stock or securities of HRB beneficially owned by the Shareholder as of the date of this Agreement over which the Shareholder has voting power, and (ii) all additional shares of common stock or other capital stock or securities of HRB of which the Shareholder acquires beneficial ownership and voting power during the period commencing on the execution and delivery of this Agreement until termination of this Agreement in accordance with Section 5.02 hereof).

NOW, THEREFORE, the parties hereto agree as follows:

ARTICLE 1

GRANT OF PROXY; VOTING AGREEMENT

Section 1.01. Voting Agreement. Shareholder hereby agrees to vote or exercise its right to consent with respect to all Shares that Shareholder is entitled to vote at the time of any vote or action by written consent to approve and adopt the Merger Agreement, the Merger, the Plan of Merger and all agreements related to the Merger and any actions related thereto (including any adjournment proposal, name-change proposal and advisory, non-binding compensation proposal set forth in the Joint Proxy Statement and submitted to the shareholders of HRB in connection with the Merger) at any meeting of the shareholders of HRB, and at any adjournment thereof, at which such Merger Agreement, Merger, Plan of Merger and other related agreements (or any amended version thereof), or such other actions, are submitted for the consideration and vote of the shareholders of HRB. Shareholder hereby agrees that, for so long as this Agreement is in effect, it will not vote any Shares in favor of, or consent to, and will vote such Shares against and not consent to, the approval of any (i) HRB Acquisition Proposal, (ii) corporate action the consummation of which would reasonably be expected to frustrate the purposes, or prevent or materially delay the consummation of, the transactions contemplated by the Merger Agreement or (iii) other matter relating to, or in connection with, any of the foregoing matters.

Section 1.02. Proxy. Shareholder hereby revokes any and all previous proxies granted with respect to Shareholder’s Shares. By entering into this Agreement, Shareholder hereby grants a revocable proxy appointing Xenith as such Shareholder’s attorney-in-fact and proxy, with full power of substitution, for and in such Shareholder’s name, to vote, express consent or dissent, or otherwise to utilize such voting power in the manner contemplated by Section 1.01 above as Xenith or its proxy or substitute shall, in Xenith’s sole discretion, deem proper with respect to Shareholder’s Shares. The proxy granted by Shareholder pursuant to this Article 1 is revocable and is granted in consideration of Xenith entering into this Agreement and the Merger Agreement and incurring certain related fees and expenses. The proxy granted by Shareholder shall be revoked upon termination of this Agreement in accordance with its terms.

Section 1.03. No Ownership Interest. Nothing contained in this Agreement shall be deemed to vest in Xenith any direct or indirect ownership or incidence of ownership of or with respect to any Shares. All rights, ownership and economic benefits of and relating to the Shares shall remain and belong to the Shareholder, and Xenith shall have no authority to manage, direct, superintend, restrict, regulate, govern or administer any of the policies or operations of HRB or exercise any power or authority to direct the Shareholder in the voting of any of the Shares, except as set forth in Section 1.02 or as otherwise expressly provided herein.

Section 1.04. Other Agreements. Prior to the termination of this Agreement in accordance with Section 5.02 hereof, Shareholder shall not enter into any agreement or understanding with any person to vote or give instructions in any manner inconsistent with this Article 1.

ARTICLE 2

REPRESENTATIONS AND WARRANTIES OF THE SHAREHOLDER

Shareholder represents and warrants to Xenith that:

Section 2.01. Authorization. Shareholder has duly executed and delivered this Agreement and the execution, delivery and performance by Shareholder of this Agreement and the consummation by Shareholder of the transactions contemplated hereby are within the powers and legal capacity of Shareholder and have been duly authorized by all necessary action. Assuming due authorization, execution and delivery by Xenith, this Agreement is a valid and binding agreement of Shareholder enforceable against Shareholder in accordance with its terms (except in all cases as such enforceability may be limited by the Enforceability Exceptions).

2

Section 2.02. Non-Contravention. The execution, delivery and performance by Shareholder of this Agreement and the consummation of the transactions contemplated hereby do not and will not (i) violate any law, rule, regulation, judgment, injunction, order or decree applicable to or binding upon the Shareholder, (ii) require any consent or other action by any Person under, constitute a default under, or give rise to any right of termination, cancellation or acceleration or to a loss of any benefit to which Shareholder is entitled under any provision of any agreement or other instrument binding on Shareholder or (iii) result in the imposition of any Lien on any asset of Shareholder.

Section 2.03. Ownership of Shares. Shareholder is the record or beneficial owner of, and has the right to vote, Shareholder’s Shares, free and clear of any Lien and any other limitation or restriction (including any restriction on the right to vote or otherwise dispose of such Shares), other than any restrictions imposed by applicable securities laws or agreements in effect as of the date hereof that have previously been made available to Xenith. None of Shareholder’s Shares is subject to any voting trust or other agreement or arrangement with respect to the voting of such Shares.

Section 2.04. Total Shares. As of the date hereof, Shareholder is the record or beneficial owner of, and has the right to vote, at least 42,398,583 Shares. Except for the Shares and the options, restricted stock units and warrants to acquire Shares owned as of the date hereof or hereafter acquired, such Shareholder does not beneficially own or have sole voting power with respect to any (i) shares of capital stock or voting securities of HRB, (ii) securities of HRB convertible into or exchangeable for shares of capital stock or voting securities of HRB or (iii) options, restricted stock units, or other rights to acquire from HRB any capital stock, voting securities or securities convertible into or exchangeable for capital stock or voting securities of HRB.

Section 2.05. Finder’s Fees. Except as provided in Section 4.7 of the Merger Agreement, no investment banker, broker, finder or financial advisor is entitled to any fees, commissions or finder’s fees from HRB in connection with this Agreement as a result of any arrangement or agreement made by or on behalf of Shareholder.

ARTICLE 3

REPRESENTATIONS AND WARRANTIES OF XENITH

Xenith represents and warrants to Shareholder that:

Section 3.01 Valid Existence; Authorization. Xenith is a corporation duly organized, validly existing and in good standing under the laws of the Commonwealth of Virginia and is a bank holding company duly registered under the Bank Holding Company Act of 1956, as amended. Xenith has the corporate power and authority to carry on its business as it is now being conducted. Xenith has full corporate power and authority to execute and deliver this Agreement and

3

to consummate the transactions contemplated hereby. The execution and delivery of this Agreement and the consummation of the Merger have been duly and validly approved by the Board of Directors of Xenith. This Agreement has been duly and validly executed and delivered by Xenith and (assuming the due authorization, execution and delivery hereof by the Shareholder) constitutes a valid and binding agreement of Xenith enforceable against Xenith in accordance with its terms (except in all cases as such enforceability may be limited by the Enforceability Exceptions).

Section 3.02 Non-Contravention. The execution, delivery and performance by Xenith of this Agreement and the consummation of the transactions contemplated hereby do not and will not (i) violate any applicable law, rule, regulation, judgment, injunction, order or decree, (ii) require any consent or other action by any Person under, constitute a default under, or give rise to any right of termination, cancellation or acceleration or to a loss of any benefit to which Xenith is entitled under any provision of any agreement or other instrument binding on Xenith or (iii) result in the imposition of any Lien on any asset of Xenith.

ARTICLE 4

COVENANTS OF THE SHAREHOLDER

Shareholder hereby covenants and agrees that so long as this Agreement is in effect:

Section 4.01. No Proxies for or Encumbrances on Shares. Except pursuant to the terms of this Agreement, Shareholder shall not, without the prior written consent of Xenith, directly or indirectly, (i) grant any proxies or enter into any voting trust or other agreement or arrangement with respect to the voting of any of Shareholder’s Shares relating to the matters set forth in Section 1.01 (other than a proxy granted to any directors or officers of HRB to vote in favor of the approval of the Merger Agreement, the Plan of Merger, and the other proposals set forth in the Joint Proxy Statement), (ii) revoke the proxy granted pursuant to Section 1.02 except upon termination of this Agreement or (iii) Transfer, encumber or otherwise dispose of, or enter into any contract, option or other arrangement or understanding with respect to the direct or indirect Transfer, encumbrance or other disposition of, any Shares, prior to the termination of this Agreement. As used herein, the term “Transfer” shall mean, with respect to any security, the direct or indirect assignment, sale, transfer, tender, pledge, hypothecation, or the gift or other disposition of such security (excluding Transfers (i) by testamentary or intestate succession or otherwise by operation of law or (ii) for estate planning purposes to an affiliate or immediate family member who agrees to be bound by the restrictions set forth in this Agreement) or any right, title or interest therein (including, but not limited to, any right or power to vote to which the holder thereof may be entitled, whether such right or power is granted by proxy or otherwise), or the record or beneficial ownership thereof; provided, that Shareholder shall not be prohibited from disposing of or

4

surrendering Shares in connection with the vesting, settlement or exercise of HRB Equity Awards for the payment of taxes thereon or, in the case of HRB Stock Options, the exercise price.

Section 4.02. Stop Transfer. Shareholder agrees with, and covenants to, Xenith that Shareholder will not request that HRB register the Transfer (book-entry or otherwise) of any certificate or uncertificated interest representing any of the Shares, unless such Transfer is made in compliance with this Agreement.

Section 4.03. Acquisition Proposals. In his, her or its capacity as a shareholder of HRB, and not in his or her capacity as a director, officer or employee of HRB, as applicable, Shareholder agrees that Shareholder will not, and will cause its officers and directors, and will instruct and use reasonable best efforts to cause its Representatives and partners (if an entity) not to, directly or indirectly, (i) initiate, solicit, knowingly encourage or knowingly facilitate inquiries or proposals with respect to a HRB Acquisition Proposal, (ii) engage or participate in any negotiations with any person concerning any HRB Acquisition Proposal, (iii) provide any confidential or nonpublic information or data to, have or participate in any discussions with or otherwise cooperate in any way with, any person in connection with any HRB Acquisition Proposal or (iv) unless this Agreement has been terminated in accordance with its terms, enter into any term sheet, letter of intent, memorandum of understanding, agreement in principle, acquisition agreement, merger agreement or other agreement (other than a confidentiality agreement referred to and entered into in accordance with Section 6.12(d) of the Merger Agreement) relating to any HRB Acquisition Proposal, in each case, except to the extent that HRB is permitted to take such action pursuant to the Merger Agreement. Shareholder will and will cause its officers, directors, and will use Shareholder’s reasonable best efforts to cause its Representatives and partners (if an entity) to, immediately cease and cause to be terminated any activities, discussions or negotiations conducted before the date of this Agreement with any person other than Xenith with respect to any HRB Acquisition Proposal. Shareholder will promptly (within twenty-four (24) hours) advise Xenith following Shareholder’s receipt of any HRB Acquisition Proposal or any inquiry which could reasonably be expected to lead to a HRB Acquisition Proposal, and the substance thereof (including the material terms and conditions of the HRB Acquisition Proposal), and will keep Xenith apprised of any related material developments, discussions and negotiations on a reasonably current basis, including any amendments to or revisions of the terms of such inquiry or HRB Acquisition Proposal, in each case to the extent HRB has not previously notified Xenith.

ARTICLE 5

MISCELLANEOUS

Section 5.01. Further Assurances. Xenith and Shareholder will each execute and deliver, or cause to be executed and delivered, all further documents and instruments and use its best efforts to take, or cause to be taken, all actions

5

and to do, or cause to be done, all things necessary, proper or advisable under applicable laws and regulations, to consummate and make effective the transactions contemplated by this Agreement.

Section 5.02. Amendments; Termination. Any provision of this Agreement may be amended or waived if, but only if, such amendment or waiver is in writing and is signed, in the case of an amendment, by each party to this Agreement or, in the case of a waiver, by the party against whom the waiver is to be effective. This Agreement shall terminate upon the earlier of (a) the Effective Time, (b) termination of the Merger Agreement in accordance with its terms or (c) if the HRB Board of Directors has delivered to its shareholders a Recommendation Change in accordance with the Merger Agreement, and upon any such termination all rights or obligations of the parties under this Agreement shall immediately terminate.

Section 5.03. Expenses. All costs and expenses incurred in connection with this Agreement shall be paid by the party incurring such cost or expense.

Section 5.04. Successors and Assigns; Obligations of Shareholder. The provisions of this Agreement shall be binding upon and inure to the benefit of the parties hereto and their respective successors and assigns; provided that no party may assign, delegate or otherwise Transfer any of its rights or obligations under this Agreement without the consent of the other parties hereto, except that Xenith may Transfer or assign its rights and obligations to any Affiliate of Xenith.

Section 5.05. Governing Law. This Agreement shall be construed in accordance with and governed by the laws of the Commonwealth of Virginia.

Section 5.06. Counterparts; Effectiveness. This Agreement may be signed in any number of counterparts, each of which shall be an original, with the same effect as if the signatures thereto and hereto were upon the same instrument. This Agreement shall become effective as between Xenith, on the one hand, and Shareholder, on the other hand, when each such party shall have received counterparts hereof signed by the other party.

Section 5.07. Severability. If any term, provision or covenant of this Agreement is held by a court of competent jurisdiction or other authority to be invalid, void or unenforceable, the remainder of the terms, provisions and covenants of this Agreement shall remain in full force and effect and shall in no way be affected, impaired or invalidated.

Section 5.08. Specific Performance; Injunctive Relief. The parties hereto agree that each party would suffer irreparable damage and that there will be no adequate remedy at law (including monetary damages) in the event any provision of this Agreement is not performed in accordance with the terms hereof and that the parties shall be entitled to specific performance of the terms hereof, injunctive relief and any other remedy to which they are entitled at law or in equity.

6

Section 5.09. Capitalized Terms. Capitalized terms used but not defined herein shall have the respective meanings set forth in the Merger Agreement.

Section 5.10. Notices. All notices, requests, claims, demands and other communications under this Agreement shall be in writing and shall be deemed given if delivered personally or sent by overnight courier (providing proof of delivery) to the parties at the following addresses (or at such other address for a party as shall be specified by like notice): (i) if to Xenith, to the appropriate address for notice thereto set forth in the Merger Agreement and (ii) if to Shareholder, to the appropriate address set forth underneath such Shareholder’s name on the signature pages hereto or such other address designated by Shareholder.

Section 5.11. Capacity. This Agreement shall apply to Shareholder solely in his, her or its capacity as a shareholder of HRB, and it shall not apply in any manner to Shareholder in any capacity as a director, officer or employee of HRB or its Subsidiaries or in any other capacity, and shall not limit or affect any actions taken by Shareholder in such other capacity.

Section 5.12. WAIVER OF JURY TRIAL. EACH PARTY ACKNOWLEDGES AND AGREES THAT ANY CONTROVERSY WHICH MAY ARISE UNDER THIS AGREEMENT IS LIKELY TO INVOLVE COMPLICATED AND DIFFICULT ISSUES, AND THEREFORE EACH SUCH PARTY HEREBY IRREVOCABLY AND UNCONDITIONALLY WAIVES, TO THE EXTENT PERMITTED BY LAW AT THE TIME OF INSTITUTION OF THE APPLICABLE LITIGATION, ANY RIGHT SUCH PARTY MAY HAVE TO A TRIAL BY JURY IN RESPECT OF ANY LITIGATION DIRECTLY OR INDIRECTLY ARISING OUT OF OR RELATING TO THIS AGREEMENT OR THE TRANSACTIONS CONTEMPLATED BY THIS AGREEMENT. EACH PARTY CERTIFIES AND ACKNOWLEDGES THAT: (I) NO REPRESENTATIVE, AGENT OR ATTORNEY OF ANY OTHER PARTY HAS REPRESENTED, EXPRESSLY OR OTHERWISE, THAT SUCH OTHER PARTY WOULD NOT, IN THE EVENT OF LITIGATION, SEEK TO ENFORCE THE FOREGOING WAIVER, (II) EACH PARTY UNDERSTANDS AND HAS CONSIDERED THE IMPLICATIONS OF THIS WAIVER, (III) EACH PARTY MAKES THIS WAIVER VOLUNTARILY, AND (IV) EACH PARTY HAS BEEN INDUCED TO ENTER INTO THIS AGREEMENT BY, AMONG OTHER THINGS, THE MUTUAL WAIVERS AND CERTIFICATIONS IN THIS SECTION 5.12.

7

IN WITNESS WHEREOF, the parties hereto have caused this Agreement to be duly executed as of the day and year first above written.